This invention relates to reduction of noise in video pixels and more particularly relates to reduction of such noise before compression.
One application of this invention is digitally removing noise from video sequences that have been digitized from the analog domain in order to increase the efficiency of a digital video compression system. A digital compression system in general takes advantage of redundant information in a video sequence in order to reduce the amount of data needed to represent the video sequence. The removal of this redundant information and subsequent coding of the video sequence produces a compressed bit stream that represents the original video sequence. The quality of the decompressed bit stream back into a video sequence depends on the ratio between the original amount of video data and the compressed bit stream data, and the efficiency with which the compression system is able to encode the information. For example, for a given sequence, the higher the compression ratio the smaller the bit stream produced. As the compression ratio increases, there is a point in which non-redundant information is degraded or lost to the compression process, therefore producing objectionable image artifacts.
In image/video compression systems, fine image details require relatively more bits to code than coarse image details, and therefore produce larger bit streams. For example, images of buildings with intricate wall details would require more bits than the clear blue sky with no clouds behind them. This fine image detail is represented as high frequency two-dimensional information; while the coarse image detail is represented as low frequency two-dimensional information that may include DC frequency, i.e. zero frequency. For purposes of this specification, it is assumed that the high-frequency detail is non-redundant and therefore necessary for a faithful rendition of the original video sequence.
Some high-frequency information is not related to actual image detail but to random noise in the original input sequence. Noise can be introduced in the video sequence in both analog and digital domains. In the analog domain, noise can be created by recording and playback of the video sequence from video tape, by errors introduced in transmission, by interference created by external sources during transmission, and other similar causes. In the digital domain, random noise can be generated by the analog-to-digital conversion process, thermal noise in components, electronic interference, etc. The two main types of noise discussed in this specification can generally be described as random. Two examples of such random noise are: random-white-gaussian-distributed noise; and random-impulsive noise. These types of noise are referred to by different names in the industry, including, ‘snow’, ‘gaussian noise’, ‘tape noise’ for the first type above; and ‘impulsive noise’, ‘salt and pepper noise’, ‘speckle noise’ for the second type.
The compression system itself has no way of knowing that some high-frequency information is random noise and therefore irrelevant to image content. If this random noise is not removed from the original video sequence, it will be coded (compressed) as part of the bit stream therefore causing degradation in quality because bits that could have been used to represent actual image information are being wasted to represent noise.
Therefore, to increase the efficiency of a digital compression system, it is desirable to reduce the amount of random noise in the original sequence so that all coded bits in the compressed bit stream represent actual picture information.
A very simple way used by prior art to reduce the high-frequency content of video sequences is the application of a low-pass filter (LFP) to an input video sequence. This LPF effectively reduces and even eliminates some high frequencies depending on the low-pass cut-off frequency characteristic. However, actual high-frequency image details are eliminated together with high-frequency noise, therefore producing a ‘soft’ picture.
Another known way to reduce random noise is to use an adaptive two-dimensional filter that preserves some high-frequency image details, like edges of objects. However, the detection of edges themselves can be affected by the noise along the edges; and depending on the low-frequency cut-off point, the images may still look soft. Furthermore, the edge detection is performed statically without regard to edge motion across time.
Other known temporal filters derive motion information from both luminance and chrominance separately, not taking advantage of the correlation between the two signals. Moreover, when other known systems reduce impulsive speckles, they use the filters in open loop mode without validation and correlation of actual impulsive spikes in the input video. Indiscriminant use of a median operator is likely to produce adverse artifacts, especially in the vertical direction.
Prior art systems have failed to recognize the utility of a median filter operation in the motion detection path which is used to eliminate impulses not in the image domain but in the motion/temporal domain. This operation makes a recursive filter perform better by controlling the value of the coefficient that controls the recursive time constant of the filter itself.
There are known temporal noise reduction systems that use motion estimation techniques instead of motion detection techniques. However, the motion estimation process is complex and does not fully guarantee the elimination of noise, just reduction.
Further limitations and disadvantages of conventional and traditional approaches will become apparent to one of skill in the art, through comparison of such systems with the present invention as set forth in the remainder of the present application with reference to the drawings.